We propose an instructions-based approach for robot programming where the programmer interacts with the robot by issuing simple commands in a scripting language, like python. Internally, these commands make use of pre-programmed motion and manipulation skills coordinated by a behaviour tree task controller. A knowledge graph keeps track of the state of the robot and the environment and of all the instructions given to the robot by the programmer. This allows to easily transform sequences of instructions into new skills that can be reused in the same or in other tasks. An advantage of this approach is that the programmer does not need to be located physically next to the robot, but can work remotely, either with a physical robot or with a digital twin. To demonstrate the concept, we show an interactive simulation of a robot manipulator in a pick and place scenario.

The growing interest in language-conditioned robot manipulation aims to develop robots capable of understanding and executing complex tasks, with the objective of enabling robots to interpret language commands and manipulate objects accordingly. While language-conditioned approaches demonstrate impressive capabilities for addressing tasks in familiar environments, they encounter limitations in adapting to unfamiliar environment settings. In this study, we propose a general-purpose, language-conditioned approach that combines base skill priors and imitation learning under unstructured data to enhance the algorithm's generalization in adapting to unfamiliar environments. We assess our model's performance in both simulated and real-world environments using a zero-shot setting. In the simulated environment, the proposed approach surpasses previously reported scores for CALVIN benchmark, especially in the challenging Zero-Shot Multi-Environment setting. The average completed task length, indicating the average number of tasks the agent can continuously complete, improves more than 2.5 times compared to the state-of-the-art method HULC. In addition, we conduct a zero-shot evaluation of our policy in a real-world setting, following training exclusively in simulated environments without additional specific adaptations. In this evaluation, we set up ten tasks and achieved an average 30% improvement in our approach compared to the current state-of-the-art approach, demonstrating a high generalization capability in both simulated environments and the real world. For further details, including access to our code and videos, please refer to https://hk-zh.github.io/spil/

Robots are more capable of achieving manipulation tasks for everyday activities than before. But the safety of manipulation skills that robots employ is still an open problem. Considering all possible failures during skill learning increases the complexity of the process and restrains learning an optimal policy. Beyond that, in unstructured environments, it is not easy to enumerate all possible failures beforehand. In the context of safe skill manipulation, we reformulate skills as base and failure prevention skills where base skills aim at completing tasks and failure prevention skills focus on reducing the risk of failures to occur. Then, we propose a modular and hierarchical method for safe robot manipulation by augmenting base skills by learning failure prevention skills with reinforcement learning, forming a skill library to address different safety risks. Furthermore, a skill selection policy that considers estimated risks is used for the robot to select the best control policy for safe manipulation. Our experiments show that the proposed method achieves the given goal while ensuring safety by preventing failures. We also show that with the proposed method, skill learning is feasible, novel failures are easily adaptable, and our safe manipulation tools can be transferred to the real environment.

What skill sets do you need to work with a world class team of artificial intelligence (AI) developers? We spoke to Misha Bilenko, the head of Yandex's machine intelligence and research (MIR) group, who told us about the trends that he's observed: "It's definitely diverse, even in terms of skill sets, but as the field is exploding you inevitably get more specialisation. Depending on the specific problem, you will need specific skills. "Some people will have the specific skills required to work only on a speech recognition system, or a recommender system, or an image classification system, and then somebody who is just trying to analyse log data will have other specific skills, as will someone doing sales prediction for a company. The developer who provides you with that top 10 list is not solving AI in the glamorous, highly technical sense of the word, but what they're building is doing an amazing amount of things "Even though we could generalise all these roles as being in the AI sub-industry, the skill set may be dramatically different between these specialities, and the differences are becoming more pronounced. "To give an example, one thing that recently has emerged is a clear distinction between data scientists and engineers, where previously the lines were somewhat more blurred.

Texas A&M University-Commerce Affiliation PLAYER SKILL DECOMPOSITION IN MULTIPLAYER ONLINE BATTLE ARENAS 2 Abstract Successful analysis of player skills in video games has important impacts on the process of enhancing player experience without undermining their continuous skill development. Moreover, player skill analysis becomes more intriguing in team-based video games because such form of study can help discover useful factors in effective team formation. In this paper, we consider the problem of skill decomposition in MOBA (MultiPlayer Online Battle Arena) games, with the goal to understand what player skill factors are essential for the outcome of a game match. To understand the construct of MOBA player skills, we utilize various skill-based predictive models to decompose player skills into interpretative parts, the impact of which are assessed in statistical terms. We apply this analysis approach on two widely known MOBAs, namely League of Legends (LoL) and Defense of the Ancients 2 (DOTA2). The finding is that base skills of in-game avatars, base skills of players, and players' champion-specific skills are three prominent skill components influencing LoL's match outcomes, while those of DOTA2 are mainly impacted by in-game avatars' base skills but not much by the other two. PLAYER SKILL DECOMPOSITION IN MULTIPLAYER ONLINE BATTLE ARENAS 3 Player Skill Decomposition in Multiplayer Online Battle Arenas Introduction Recently a unique type of sports, namely electronic sports (eSports), emerges as a popular genre of computer games, in which human players compete with one another in online, simulated environments governed by rules and regulations similar to those found in traditional forms of sports. A recent report released by SuperData (2016) showed that the worldwide market for eSports, by the end of 2015, has reached approximately 748 million dollars and is expected to grow to 1.9 billion dollars by 2019. Each team consisting of five players has a base to defend and the goal is to attack the opposite teams' champions and ultimately destroy the opponent's base.